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REMARKS/ARGUMENTS 

Reconsideration of this application is requested. Claims 1 to 20 are in the case. 
Claim Rejections - 35 USC S 102 and S 103 

The Examiner has rejected claims 1-3, 5-10, and 12-20 as being anticipated by U.S. 
patent No. 6,728,401 (Hardeberg). The Examiner has rejected claims 4 and 11 as 
being obvious in view of Hardeberg in light of U.S. Patent 5,748,764 (Benati). These 
rejections are respectfully traversed, and it is respectfully submitted that the present 
invention as claimed is patentable over the teachings of Hardeberg and Benati. 

Background and Summary of the Invention 

The present invention involves correcting a red-eye effect in a digital image provided by 
a cluster of pixels. In a flow chart, Figure 1 lays out different steps that can be followed 
to achieve this result. As shown in Figure 1, after the digital photograph image is 
initially provided in step 22, in step 24 this digital image is analyzed to segment small 
compact red objects within the digital image. The segmentation is performed using at 
least one tophat operation performed on the image to obtain a tophat image. The 
tophat operation is conducted to produce a tophat image in which concentrations of red 
are highlighted. Bright regions in the tophat image will correspond to concentrations of 
red in the original image. The resulting tophat image is then subjected to an intensity 
threshold to identify regions of sufficient brightness to become objects of interest. Of 
these objects, only objects with compactness less than a compactness threshold are 
retained as objects of interest, such that elongated patches of redness are removed to 
produce a segmentation mask. In this segmentation mask, segmented objects are 
represented by white pixels and all of the other pixels are black. Subsequently, features 
are extracted from the digital image for each object, and the segmented object is then 
classified based on the abstracted feature set. In particular, each segmented object is 
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then classified based on the abstracted feature set. More particularly, each segmented 
object is classified based on the probability that the object is a red-eye defect. 

Telephone Interview Summary 

On August 30, 2007, the applicant Rod Hale, the undersigned, and Examiners Wu and 
Patel had a telephone interview regarding both this draft response and a petition filed 
herewith. Draft copies of both the response and the petition were provided to the 
Examiners prior to the telephone interview. 

The petition relates to vacating the finality of the Office Action dated July 2, 2007. 
During the telephone interview, the undersigned discussed with the Examiners why the 
Office Action of July 2, 2007, should not have, in his respectful submission, been made 
final. Examiner Wu requested the applicant to submit the petition with the response 
after final. Accordingly, the petition has been filed. 

The undersigned, Mr. Hale, and Examiners Wu and Patel also discussed the Office 
Action dated July 2, 2007, the Hardeberg reference cited therein, and the draft response 
to the Office Action submitted to the Examiners. Examiner Wu requested that copies of 
the relevant pages of prior publications providing definitions of "tophat operation" be 
provided. Copies of these documents have accordingly been electronically transmitted 
with this response. During the telephone interview, Examiner Wu also commented that 
Hardeberg discloses erosion and dilation operation. In response, the undersigned, 
commented that these erosion and dilation operations are part of the morphological 
filtering steps, and referred the Examiners to the response to the first Office Action, 
which dealt with this position in detail. 

Detailed Reply to Claim Rejections 

The Examiner takes the position that Hardeberg anticipates claim 1 of the present 
application. Specifically, the Examiner takes the position that the process of defining a 
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region of interest (ROI) described in Hardeberg corresponds to the tophat operation 
claimed. This position is respectfully traversed. According to Hardeberg, a ROI may 
be defined manually by a user selecting a border around the portion of the color image 
containing one or more artifacts, or through automated methods, such as known neural 
network techniques for detecting faces and eyes (Column 4, Lines 1-6). Accordingly, 
the process of defining an ROI can be described as selecting a portion of an original 
image that contains one or more artifacts. This is usually done to reduce the size of the 
image to be processed and/or to reduce the likelihood of the system incorrectly 
identifying an artifact in the non-artifact portions of the image. For the following 
reasons, it is respectfully submitted that the process of defining an ROI falls outside of 
the scope of the tophat operation claimed in the present application. 

Claims 1. 8 and 17 

Plain meaning of "tophat operation" and "tophat image" 

Claim 1 recites "conducting at least one tophat operation over each pixel in the digital 
image to provide a tophat image". As specified at section 21 1 1 .01 of the MPEP, during 
"examination, the claims must be interpreted as broadly as their terms reasonably 
allow.... This means that the words of the claim must be given their plain meaning 
unless the plain meaning is inconsistent with the specification." 

In the specification of the present application, the term tophat operation, whether a dark 
tophat operation or a bright tophat operation, is used to refer to an operation for 
segmenting an image based on defined intensity thresholds to highlight concentrations 
of red: high intensity thresholds in the case of bright tophat operations and low intensity 
thresholds in the case of dark tophat operations. After this segmentation, the resulting 
image is different from the starting image, unlike the region of interest disclosed by 
Hardeberg, which is merely a portion of the original image. 

Section 2111.01 of the MPEP also provides that the "ordinary and customary meaning 
of a claim term is the meaning that the term would have to a person of ordinary skill in 
the art in question at the time of the invention, i.e., as of the effective filing date of the 
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patent application." In the specification of the present application, the terms "tophat 
operation" and "tophat image" have meanings that are consistent with their general 
meanings in the art (and are inconsistent with the Hardeberg's description of how the 
ROI is defined). That is, the specification itself cites a reference providing a definition of 
tophat operation - see paragraph 31 referring to "Digital Image Processing" by Rafael 
C. Gonzalez and Richard E. Woods, Addison Wesley Publishing, 1992 edition) (copies 
of the relevant portion of this reference are attached as Appendix A). This reference, on 
pages 556 and 557, defines a top-hat transformation more generally as follows: "The 
so called morphological top-hat transformation of an image, denoted h, is defined as h = 
f - (f o b) where, as before, f is the input image and b is the structural element function. 
This transformation - which owes its original name to the use of the cylindrical or 
parallelepiped structuring element function with a flat top - is useful for enhancing detail 
in the presence of shading, [where] o in the above equation is known as an opening 
operation which is an erosion followed by a dilation". 

Elsewhere in the art (see Beucher, S. [1991], "The watershed transformation applied to 
image segmentation". Conference on Signal and Image Processing in Microscopy and 
Microanalysis, pp. 299—314), "The Top-Hat transform TH(f) defined as the difference 
between the function and its morphological opening is a contrast detector suitable for 
enhancing in the image the blazing zones" (see page 17 of the paper attached as 
Appendix B). The blazing zones are the bright spots in the image (for or redeye 
example it is the red regions in the R-G image). 

Based on the foregoing, a tophat operation could be generally defined as an operation 
defined over a digital image to detect and enhance contrast. Similarly, a tophat image 
could be generally defined as a image in which areas of contrast have been enhanced. 
However, the step of defining a ROI described by Hardeberg will not necessarily identify 
areas of contrast, and the ROI selected will certainly not enhance these regions of 
contrast, as the ROI selected will simply be a portion of the original digital image. 
In view of the foregoing it is respectfully submitted that the selection of a ROI, as 
described by Hardeberg, does not fall within the scope of "conducting at least one 
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tophat operation over each pixel in the digital image to provide a tophat image", as 
recited by claim 1. That is, selecting a ROI is not an example of a tophat operation, nor 
is the ROI itself an example of a tophat image. If the Examiner elects to maintain this 
rejection, then clarification is request. Specifically, the Examiner is respectfully 
requested to clarify why selecting a ROI is an example of a tophat operation, and why 
the ROI itself is an example of a tophat image, given that this usage is inconsistent with 
the meaning of tophat operation and tophat image as these terms are used both in the 
specification and the art generally. 

The tophat operation claimed is performed on each pixel of the image 

The tophat operation as defined in the claim involves performing an operation on each 
pixel of the image to produce a tophat image. In contrast, Hardeberg does not disclose 
defining a ROI by performing an operation on each pixel of the image. 

Column 3 Lines 63-67 and Column 4 Lines 1-6 of Hardeberg read as follows: 

"In the first block 220, a region of interest (ROI) is optionally defined. Defining a ROI 
reduces the amount of processing needed. It also reduces the probability that the 
artifact detection program will identify and modify portions of the, image that resemble 
reflected light artifacts (such as red eye) but that are not. A ROI may be defined 
manually by a user selecting a border around the portion of the color image containing 
one or more artifacts. The ROI may also be defined through automated methods, such 
as known neural network techniques for detecting faces and eyes." 

Nothing in this passage suggests defining a ROI by performing an operation over each 
pixel in the image. Indeed, given one of the stated purposes of defining the ROI - to 
reduce the amount of processing needed - defining a ROI by performing an operation 
over each pixel in the image appears to be generally inconsistent with the teachings of 
Hardeberg as performing an operation over each pixel in the image would be 
processing intensive. In addition and unlike a ROI, the image resulting from performing 
an operation over each pixel in the image would be the same size as the original image. 
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Accordingly, it is respectfully submitted that the selection of a ROI, as described by 
Hardeberg, does not fall within the scope of "conducting at least one tophat operation 
over each pixel in the digital image to provide a tophat image", as recited by claim 1, as 
the selection of a ROI described by Hardeberg does not appear to involve performing 
an operation over each pixel in the digital image. Indeed, performing an operation over 
each pixel in the digital image seems contrary to the Hardeberg's objective of reducing 
the size of the image to reduce the processing required. If the Examiner elects to 
maintain this rejection, then clarification is requested. Specifically, the Examiner is 
respectfully requested to clarify how Hardeberg is being read to describe selecting a 
ROI by conducting at least one operation over each pixel in the digital image. 

For analogous reasons it is respectfully submitted that system claim 8 and computer 
product claim 17 and their corresponding dependent claims are not anticipated by 
Hardeberg. 

Claims 2 and 3 

The Examiner also takes the position that Hardeberg anticipates claim 2 of the present 
application. Specifically, the Examiner takes the position that the steps of dilating and 
eroding of the second structuring element described in Hardeberg (Column 8, Lines 38- 
49) correspond to the steps of greyscale dilation and erosion of the digital image of 
claim 2. This position is respectfully traversed. The greyscale dilation and erosion 
steps of claim 2 form part of the tophat operation of claim 1. Accordingly, since the 
Examiner takes the position that the process of defining a region of interest described in 
Hardeberg corresponds to the tophat operation of claim 1, the steps of dilating and 
eroding of the second structuring element in Hardeberg must form part of the process of 
defining a region for Hardeberg to anticipate claim 2 of the present application. 
However, the steps of dilating and eroding of the second structure referred to by the 
Examiner do not form part of the process of defining a region of interest, but are in fact 
performed after the process of defining a region of interest. This is clear from the fact 
that the dilation and erosion are performed using the second structuring element and 
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the second structuring element has "a shape of a square with sides of approximately 
1/10 of the width of the ROI" (Column 8, Lines 37-39). Therefore is respectfully 
submitted that Hardeberg does not anticipate claim 2 of the present application. 

Please also note that claim 2 further defines the tophat operation initially defined in 
claim 1 to be performed over each pixel in the digital image. The steps of eroding and 
dilating the structuring elements described in Hardeberg are not performed over each 
pixel in the digital image, but merely over a portion of the region of interest initially 
defined. Further, it is respectfully submitted that it would not have been obvious to 
modify the teachings of Hardeberg to provide this erosion and dilation during the step of 
defining a region of interest. Specifically, as described above, the steps of eroding and 
dilating are applied only to the structuring element, which is a portion of the ROI. There 
does not appear to be any motivation to apply the steps of eroding and dilating various 
pixels during selection of the ROI, as it is not clear how eroding and dilating in the 
context of Hardeberg, which does not disclose a tophat operation, would facilitate 
selection of the ROI. Further, applying the erosion and dilation to every pixel of the 
digital image would seem to obviate the purpose served by selecting a region of interest 
in Hardeberg, which is to reduce the amount of processing required. 

For analogous reasons it is respectfully submitted that system claim 9 and computer 
product claim 18 are neither anticipated nor rendered obvious by Hardeberg. 

The Examiner also takes the position that Hardeberg anticipates claim 3 of the present 
application. Specifically, the Examiner takes the position that the steps of eroding and 
dilating of a structuring element described in Hardeberg (Column 8, Lines 25-37) 
correspond to the steps of greyscale erosion and dilation of the digital image of claim 3. 
This position is respectfully traversed. The greyscale dilation and erosion steps of claim 
3 form part of the tophat operation of claim 1. Accordingly, since the Examiner takes 
the position that the process of defining a region of interest described in Hardeberg 
corresponds to the tophat operation of claim 1, the steps of dilating and eroding of the 
structuring element in Hardeberg must form part of the process of defining a region for 
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Hardeberg to anticipate claim 3 of the present application. However, the steps of 
eroding and dilating of the structuring element referred to by the Examiner do not form 
part of the process of defining a region of interest, but are in fact performed after the 
process of defining a region of interest. This is clear from the fact that the dilation and 
erosion is performed using the structuring element and in Column 8, Lines 27-30 states 
that the structuring element has "a shape of a square with sides of approximately 1/30 
of the width of the ROI." Therefore, is respectfully submitted that Hardeberg does not 
anticipate claim 3 of the present application. 

Similar to the foregoing discussion with respect to claim 2, please note that claim 3 
further defines the tophat operation initially defined in claim 1 to be performed over each 
pixel in the digital image. The steps of eroding and dilating the structuring elements 
described in Hardeberg are not performed over each pixel in the digital image, but 
merely over a portion of the region of interest initially defined. Further, it is respectfully 
submitted that it would not have been obvious to modify the teachings of Hardeberg to 
provide this erosion and dilation during the step of defining a region of interest. 
Specifically, as described above, the steps of eroding and dilating are applied only to 
the structuring element, which is a portion of the ROI. There does not appear to be any 
motivation to apply the steps of eroding and dilating various pixels during selection of 
the ROI, as it is not clear how eroding and dilating in the context of Hardeberg, which 
does not disclose a tophat operation, would facilitate selection of the ROI. Further, 
applying the erosion and dilation to every pixel of the digital image would seem to 
obviate the purpose served by selecting a region of interest in Hardeberg, which is to 
reduce the amount of processing required. 

For analogous reasons it is respectfully submitted that Hardeberg does not anticipate or 
render obvious system claim 10 and computer product claim 19. 

Amendments to the claims 

Former claims 5, 12 and 20 depended from claims 1, 8 and 17 respectively. Further, 
claims 5, 12 and 20 all recite a "bright tophat image"; however, claims 1, 8 and 17 do 
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not provide an antecedent basis for this term. Accordingly, claims 5, 12 and 20 have 
been amended to depend from claims 3, 10 and 19 respectively, all of which claims 
provide an antecedent basis for the term "bright tophat image". 

In view of the foregoing, it is respectfully submitted that the claims as amended are 
allowable over the cited references. Allowance of the application is respectfully 
requested. 

Additional Arguments 

As outlined in more detail in the petition filed herewith, it is submitted that the Examiner 
prematurely issued a final rejection under MPEP § 706.07(a). In general, the Examiner 
introduced new grounds of rejection when he changed his position regarding what 
elements described by Hardeberg satisfy the limitations of the claims. 

Regardless of the finality of the action, however, it is respectfully submitted that the 
claims clear the art cited. Accordingly, allowance of the application is respectfully 
requested. 

If any questions arise, it is requested that the undersigned be contacted at the number 
provided below. 



Respectfully submitted, 



BERESKIN & PARR 




Ian C. McMillan 
Registration No. 43,390 
Telephone: (416) 957-1644 
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